Search for: All records

Introduces OAT (Offline with Augmented Trajectories), a generative sub-trajectory augmentation method designed to enhance off-policy evaluation accuracy. Experiments across robotics, healthcare, and e-learning show substantial performance gains over baselines. 

In the realm of reinforcement learning (RL), off-policy evaluation (OPE) holds a pivotal position, especially in high-stake human-centric scenarios such as e-learning and healthcare. Applying OPE to these domains is often challenging with scarce and underrepresentative offline training trajectories. Data augmentation has been a successful technique to enrich training data. However, directly employing existing data augmentation methods to OPE may not be feasible, due to the Markovian nature within the offline trajectories and the desire for generalizability across diverse target policies. In this work, we propose an offline trajectory augmentation approach, named \textbf{OAT}, to specifically facilitate OPE in human-involved scenarios. We propose sub-trajectory mining to extract potentially valuable sub-trajectories from offline data, and diversify the behaviors within those sub-trajectories by varying coverage of the state-action space. Our work was empirically evaluated in a wide array of environments, encompassing both simulated scenarios and real-world domains like robotic control, healthcare, and e-learning, where the training trajectories include varying levels of coverage of the state-action space. By enhancing the performance of a variety of OPE methods, our work offers a promising path forward for tackling OPE challenges in situations where human-centric data may be limited or underrepresentative. 

Sub-Terahertz (THz) frequencies between 100 GHz and 300 GHz are being considered as a key enabler for the sixthgeneration (6G) wireless communications due to the vast amounts of unused spectrum. The 3rd Generation Partnership Project (3GPP) included the indoor industrial environments as a scenario of interest since Release 15. This paper presents recent sub- THz channel measurements using directional horn antennas of 27 dBi gain at 142 GHz in a factory building, which hosts equipment manufacturing startups. Directional measurements with copolarized and cross-polarized antenna configurations were conducted over distances from 6 to 40 meters. Omnidirectional and directional path loss with two antenna polarization configurations produce the gross cross-polarization discrimination (XPD) with a mean of 27.7 dB, which suggests that dual-polarized antenna arrays can provide good multiplexing gain for sub-THz wireless systems. The measured power delay profile and power angular spectrum show the maximum root mean square (RMS) delay spread of 66.0 nanoseconds and the maximum RMS angular spread of 103.7 degrees using a 30 dB threshold, indicating the factory scenario is a rich-scattering environment due to a massive number of metal structures and objects. This work will facilitate emerging sub-THz applications such as super-resolution sensing and positioning for future smart factories.